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ABSTRACT 

Suppose  one  is  able  to  observe  sequentially  a  series  of 
independent  observations  ,  such  that  X^ ,  X£ , . . . ,  Xy  ^  are 

i.l.d.  with  known  density  fg  and  X^.X^^ .  are  i.i.d.  with  den¬ 

sity  fg  where  v  is  unknown.  Define 

„{0}  “  ”  f0(V 

n  k«l  i-k  W 

It  is  known  that  rules  which  call  for  stopping  and  raising  an  alarm 

{0} 

the  first  time  n  that  or  a  mixture  thereof  exceeds  a  prespeci¬ 

fied  level  A  are  optimal  methods  of  detecting  that  the  density  of  the 
observations  is  not  f  q  any  more. 

Practical  applications  of  such  stopping  rules  require  know¬ 
ledge  of  their  operating  characteristics,  whose  exact  evaluation  is 
difficult.  Here  are  presented  asymptotic  (A  -*•  <*>)  expressions  for  the 
expected  stopping  times  of  such  stopping  rules  (a)  when  v  ■  •  and  (b) 
when  v  •  1.  We  assume  that  the  densities  fg  form  an  exponential 
family  and  that  the  distribution  of  log(fg(X^)/fg(X1))  is  (strongly) 
non-lattice. 

Monte  Carlo  studies  indicate  that  the  asymptotic  expressions 
are  very  good  approximations  even  when  the  expected  sample  sixes  are 

small. 
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I.  INTRODUCTION 


Suppose  one  accumulates  Independent  observations  from  a  certain 
process.  Initially,  the  process  Is  at  State  #0.  At  some  unknown  point 
In  time  something  occurs  (e.g.,  a  "breakdown")  which  puts  the  process 
in  State  #1,  and  consequently  the  stochastic  behavior  of  the  observa¬ 
tions  changes.  It  is  of  interest  to  declare  that  a  change  took  place 
(to  "raise  an  alarm")  as  soon  as  possible  after  its  occurrence,  subject 
to  a  restriction  on  the  rate  of  false  detections.  It  is  assumed  that 
the  aforementioned  observations  are  the  only  information  one  has  about 
the  process,  and  the  problem  is  to  construct  a  good  detection  scheme. 

Practical  examples  of  this  problem  arise  in  areas  such  as  health, 

quality  control,  ecological  monitoring,  etc.  For  instance,  consider 

surveillance  for  congenital  malformations  in  newborn  infants.  Under 

normal  circumstances,  the  percentage  of  babies  born  with  a  certain  type 

of  malformation  has  a  known  value,'pQ9^  Should  something  occur  (such  as 

t 

an  environmental  change,  the  introduction  of  a  new  druj*  to  the  market, 
etc.)  the  percentage  may  increase,  (eTgT,  jthe  thalldomlde  eplsode  nf  the"9 
I960 *-»)  .>>  One  would  want  to  raise  an  alarm  as  quickly  as  possible  after 
a  change  would  have  taken  place,  subject  to  an  acceptable  rate  of  false 
alarms.  Generally,  the  problem  arises  wherever  surveillance  is  being 


done. 


r\ 


A  solution  to  the  problem  depends  on  what  is  known  in  advance 
about  the  distributions  of  the  observations.  Let  f^  denote  the  density 
of  observations  with  respect  to  a  o-finite  measure  y  when  the  process 
1b  in  State  #0,  let  fg  denote  the  density  of  observations  with  respect 
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to  p  when  the  process  is  in  State  #1,  and  let  v  denote  the  unknown 
point  in  time  when  the  first  observation  from  State  #1  is  made.  Thus 
one  has  a  sequence  of  independent  observations  X^ .Xj, • . . ,  such  that 
X^^, . . .  ,X^_^  are  i.i.d.  with  density  fg  and  X^X^^,...,  are  i.i.d. 
with  density  f g  where  1  <_  v  £  00  is  unknown.  It  will  be  assumed  here 
that  f Q9f q  belong  to  an  exponential  family  of  distributions  and  that 
fg  is  known. 

Solutions  for  the  problem  which  are  in  current  use  are  known 
as  CUSUM  procedures.  For  a  survey  see,  for  instance,  Johnson  and  Leone 
(1962).  (See  also  Weatherall  and  Haskey  (1976).)  Lorden  (1971)  proved 
a  first-order  asymptotic  optimality  property  of  a  certain  class  of  pro¬ 
cedures  for  reacting  to  a  change  in  distribution.  When  fg  is  known, 
this  class  includes  most  of  the  standard  appropriate  CUSUM  techniques 
as  special  cases.  When  fg  is  unknown,  Lorden  (1971)  suggests  a  first- 
order  asymptotically  optimal  procedure.  (Asymptotic  operating  charac¬ 
teristics  of  this  and  related  procedures  are  given  in  Poliak  and  Siegmund 
(1975).  Further  refinements  can  be  obtained  using  results  of  Lai  and 
Siegmund  (1977) .) 

Shiryayev  (1963,  1978)  solved  the  problem  in  a  Bayesian  frame¬ 
work  in  the  case  that  fg  is  known. 

An  optimal  solution  in  a  classical  framework  is  presented  in 
Poliak  (1983).  Asymptotic  operating  characteristics  of  this  and  related 
procedures  are  the  subject  under  study  here. 

Without  loss  of  generality,  let  the  assumed  exponential  family 
be  defined  by 

fy(x)  -  ,  ,  ,  (1 


2 


where  ft  is  an  interval  on  the  real  line,  0  -  i|>(0)  -  tp*  (0) .  Let  F  be 
a  probability  measure  on  ft  with  F((0))  *  0.  Let  0  <  A  <  00 .  Define 

{vi  °  )  n  yZ  “  X  -(n-k4-l)^Cy) 

Rly'  -  Z  H  /rY\  -  Z  e  1  K  1 

n  k-1  i-k  t0'i^  k“l 

RF  -  /  dF(y) 

n[y*  -  min{n|R*yJ  >  A) 
a  n  — 

NF  -  min{n|RF  >  A)  . 

A  n  — 

{0} 

Raising  an  alarm  at  time  is  an  optimal  procedure  when  the  value 

0  (of  the  parameter  of  the  distribution  after  a  change  occurred)  is 

p 

known  and  raising  an  alarm  at  time  has  optimality  properties  when 
6  is  unknown  (Poliak  (1983)). 

In  order  to  evaluate  and  compare  between  procedures  one  needs 
to  formalize  a  restriction  on  false  detections  as  well  as  to  formalize 
an  expression  for  the  speed  of  detection  of  a  change  after  its  occur¬ 
rence.  The  restriction  on  false  detections  is  usually  formalized  as 
a  requirement  that  the  expected  number  of  observations  until  a  false 
alarm  (assuming  that  v  ■  °»)  exceed  a  prespecified  value  B.  This 
suggests  a  need  for  evaluating  E(N^y^|v  -  «),  E(NF|v  ■  °°) .  The  quality 
of  a  procedure  with  regard  to  the  speed  of  detection  of  a  change  after 
its  occurrence  is  often  measured  by  the  supremum  (or  essential  supremum) 
of  the  expected  number  of  observations  that  it  takes  to  detect  a  change 
after  its  occurrence,  given  that  no  false  alarms  have  previously  been 
raised  (see  Lorden  (1971),  Poliak  and  Siegmund  (1975)).  This  suggests 
a  need  for  evaluating  E(N^  -  v|v  -  1,  0),  E(NF  -  v|v  ■  1,  8).  These 
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operating  characteristics  are  difficult  to  compute.  For  simulations 
see  Roberts  (1966) . 

In  this  article,  asymptotic  expressions  (A  -*•  °°)  for  these 
operating  characteristics  are  presented.  Monte  Carlo  studies  indi¬ 
cate  that  these  expressions  are  very  good  approximations  even  when 
the  expected  samples  sizes  are  small. 


II.  THE  AVERAGE  RUN  LENGTH  WHEN  V  -  « 

(y)  (v) 

Denote  by  ,  E^J  the  probability,  expectation  respectively 
when  1  V  <_ «°,  X^,...,X^_^  are  i.i.d.  with  density  and  are  independent 
of  X^.X^^,...,  which  are  i.i.d.  with  density  f^.  Let  Pq*eq  denote 
probability,  expectation  respectively  when  V  -  “>.  Let  F  be  a  prob¬ 
ability  measure  on  H  with  F({0})  -  0.  Denote 


1(0) 


log 


w 

f0(V 


F(0)  7{0) 

Ei  h 


y\  -  'PCy ) 


e  z{9} 
Eozi 


■  mintnll^^  B),  *  “  if  no  such  n  exists 

(  l  “fOi  Z\  ~  ®] 

Cj  -  1/lim  Ejy)  e  '  i*1  1  ' 

U  B-*«>  1 

cj  -  1/ /(1/Cq)  dF(y)  . 

y  f 

The  computations  of  C'  and  are  applications  of  renewal  theory  and 
have  been  calculated  in  other  contexts.  (See  Siegmund  (1975),  Lai  and 
Slegmund  (1977),  Theorem  6.2  of  Woodroofe  (1982).) 


THEOREM  1.  (1)  E„  N:1^  >  A  for  «11  y  <  R.  If  I(y)  <  ”,  then  for  any 

<V  (y)  <V 

An  >  0  there  exists  a  constant  0  <  C  <  00  such  that  E_  N:*  <  C  A 

0  y  0  A  —  y 

whenever  A  ^  Aq. 

(ii)  If  y  e  !i,  I(y)  <  «  and  the  pj^-distribution  of 
log(fy(X^)/fp(X^))  is  non-lattice,  then 
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non- 


Eo  NlyJ  "  A  C0  (1  +  o{1))  » 

where  o(l)  ♦  0  as  A  4  °°. 

THEOREM  2.  Suppose  that  the  P^-distribution  of  is  strongly 
lattice  (see  Stone  (1965))  for  all  y  e  Then 

(i)  E.  NE  >  A.  If  F({y | I(y)  <  “})  -  1,  then  for  any  A-  >  0 

u  A  —  A  A 

A0  F  “o 

there  exists  a  constant  0  <  C^,  <  00  such  that  Eg  <_  whenever 

A  -  Ao* 

(ii)  If  F({y | I(y)  <  “))  -  1,  then 

Eg  nJ  -  A  Cg  (1  +  o(l))  , 
where  o(l)  +  0  as  A  + 
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III.  PROOFS 


The  proof  of  Theorems  1  and  2  Is  based  on  the  observation  that 

p 

(under  Pn)  R  -  n  is  a  martingale  with  zero  expectation  with  respect 


to  g(X, ),  so  that  for  stopping  times  N  which  are  well-behaved 
1  n 


V  - 

of  vV 


The  proof  becomes  an  analysis  of  the  asymptotic  behavior 


N 


Si 


or  any  m,  r 


(1) 


nri-r 


r  m+r  {y}  m  (yl 

1  £  e1"1  1  dF(y) 


m+r 

+  /  Z 

k=nri-l 


k=l 

_nrt-r  {y} 

e  i=k  1  dF(y)  . 


{y} 

Make  note  of  the  following  three  observations:  (I)  Since  <  0, 

the  first  expression  on  the  right  side  of  equation  (1)  becomes  negli¬ 
gible  as  r  becomes  large.  (II)  The  second  expression  on  the  right 

side  of  equation  (1)  when  regarded  as  a  process  in  r  has  the  same 

F 

stochastic  P„-behavior  as  the  original  process  R  .  (Ill)  If  the 
u  n 

F  F 

value  of  R  is  large,  the  process  R  behaves  approximately  like  the 
in  n 

first  expression  on  the  right  side  of  (1)  for  n  ■  mfr  closely  follow¬ 
ing  m. 

The  idea  of  the  proof  can  now  be  described  as  follows.  Let  c 
be  a  large  constant,  and  let  A  be  much  larger  than  c.  Regard  the 
stopping  time  N*  which  at  first  tells  one  to  continue  sampling  until 
Nf.  .  If  "soon"  thereafter  RF  >  A,  let  N.  *  If  not,  forget  the 

A/C  n  —  A  A 

F  F 

first  c  obse  vations  and  reapply  to  the  sequence  of  observa- 

F  F 

tions  iuj  owx.ig  Repeat  this  until  the  first  time  that  Rr  A 
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F  * 

"soon"  after  NA^.  This  first  time  defines  NA.  By  virtue  of  observa- 

F 

tion  (I)  it  will  be  shown  that  the  asymptotics  of  EnR  F  are  the  same 

F  aA 

as  those  of  EJR  . . 

^  F 

The  repeated  applications  of  NA^c  (conditional  on  their  exis¬ 
tence)  will  be  shown  to  be  approximately  independent  of  each  other.  By 

p 

virtue  of  observations  (I)  and  (II),  it  will  be  shown  that  E.R  *  is 

°»! 

approximately  equal  to 

(2)  E-(RF. lRF  >  A  "soon"  after  the  first  application  of  NF,  )  . 

0  n  —  A/  c 

NA 

P 

Letting  m  *  NA^C  In  equation(l),  note  that  the  first  expression 

p 

on  the  right  side  of  equation  (1)  is  equal  to  R  x  w  where 

n  N  +r 

nF  NA/c  *A/c 

_  NA/c+r  _{y} 

i=N'  ,  +1  1 

W  F  -  /  e  A/C  dF  (y)  , 

N.  /  +r 
A/c 


NA/c  Z{y) 

dF  (y)  =  l  e  dF(y)/R* 

k=1  NA/c 


By  virtue  of  observation  (III)  it  will  be  shown  that  {R  >  A  "soon" 

n 

F  i 

after  the  first  application  of  NA/c;  is  approximately  equal  to 

(RFp  W  >_  K  for  some  1  r  <  00 } .  Let  H  =  tnin(r|RF  W  „ 

Na/  K'.+r  IT,  Na/  +r 

A/c  A/c  A/c  A/c 

*  w  if  no  such  r  exists.  It  follows  from  (2)  that 


>  A), 


VF*  appr<”,1“tely  “ 

u  s.  n:  ,  i 


F  H1  <  -> 

NA/c  NA/c+H1 


E0(R  F  W  F  .  u  <  <*») 

0  NF  Na/  +H.’  H1  1 

_ A/c  A/c  1 _ 

VH1  < ro) 
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:•  ~  'V 


Conditional  on  3(X  „  ),  W  is  a  P.-martingale  (with  unit 

J.  v.f  „r  .  U 

NA/c  NA/c  r 

expectation  with  respect  to  3(X  _  ).  Therefore  the 

NA/c+1  NA/c+r 

F 

numerator  in  (3)  is  equal  to  E-R  _ 

0  NT 
NA/c 

Results  of  Lai  and  Siegmund  (1977)  yield  that,  with  K(y)  =  l/C^, 


P0(Hi<H3(X1,...,X  »=P0<W  ,_>A/R 


A/c 


NA/c+r 


N 


A/c 


for  some  l<r<CD|3(X . X  )) 

NI/ 

A/c 

approximately  ;  K(y)  dFl(y) 


A/R 


A/c 


Therefore 


(4)  Pq(H1< 


approximately  . 

'  E0A~  f  K(y)  Rp  dF^y) 

\/e 

-  A'1  /  K(y)  Eq(RFf  dFj (y) ) 

NA/c 

=  A"1  /  K(y)  EqNF/c  dF (y) 

V/ 

-  - ^  f  K(y)  dF(y) 


where  the  equality  (4)  follows  from  the  definition  of  dF^(y)  and  the 
_  n  7  y 

fact  that  £  e  -  n  is  a  P  -martingale  (with  zero  expectation) 

k-1 

with  respect  to  3(X, ,...,X  ).  It  now  follows  from  (3)  that 
l  n 


vl  •  VFF  1  A//  K(y)  dF(y) 

N* 

which  is  the  heart  of  the  content  of  Theorem  2. 


I 


The  turning  of  these  heuristic  arguments  into  a  rigorous  proof 
requires  the  ten  lemmas  presented  in  the  sequel.  The  method  involved 
is  linear  and  nonlinear  renewal  theory  (cf.  Feller  (1971),  Stone  (1965), 
Woodroofe  (1976),  Lai  and  Siegmund  (1977)),  For  a  survey  see  Woodroofe 
(1982). 


PROOF  OF  THEOREM  l(i),  THEOREM  2(i).  Note  that  under  PA  both  R{y} 


n 


-  n 


and  Rn-n  are  martingales  with  zero  expectation  with  respect  to 
3(X^, . . . .X^) .  Denote 

ir*y^  *  min{n|  max  exp{  E  zfyH  >  a! 

A  1  k=l , . . . ,  n  4=k  1  >  ~  J 

it?  »  min(n|  max  /  exp(  E  Z^yH  dF(y) >  a}  . 

1  k=l, . . .  ,n  4=k  1  •  ~  > 

It  is  well  known  that  EQ  <  «,  eq  irF  <  »  (cf.  Lorden  (1971)). 

Since  N^y^  <_  ir£y^  and  NF  _<  ttF  it  follows  that  E^  N^y^  <  »  and 
E0  <  <  <”•  Hence  EQ(R^}-N^y})  and  E()(R^f-nJ)  exist.  Since 

|R^y^ | <  A,  |rF| <  A  on  {N^y)>n},  (N^  >  n)  respectively,  it  is 


n 

see  that 


easy  to 


lim  inf  |R  -n]dPn  -  0  ,  lim  inf  ]  |RF-n|dPr  -  0 

—  V>>">  >'■* -  "  0 

A 


Hence  by  the  martingale  optional  stopping  theorem  (cf.  Chow,  Robbins, 

Siegmund  (1971),  Theorem  2.3  (p.  23))  En(R^?\  -  n[y^)  -  0  and 

0  „lyl  A 

A 
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H 


E0(RS"  NA)  *  °*  Therefore*  E0  N{?y}  >  A  and  E0  -  EQ  RF  >  A. 

"a  V  \ 


This  completes  the  proof  of  the  first  parts  of  Theorem  l(i)  and 
Theorem  2(i). 


For  the  second  part  of  Theorem  l(i),  let  SQ  =  0  and  define 


recursively  for  i  >  1  by 


S  -  min(n| n  >  S  ,  Z  z{y}  i  (0,  log  A)} 

J“Si-l+1  J 


Then  T^y}  <  Z^  S±  where  C  =  min{i|Z  1  +1  z!y}  e  [log  A,  log(2A)]} 


By  Wald's  lemma, 


S0  -  E0S1/P0  (  1  Z!y>  €  [1°S  A*  lo8  (2A) ] )  . 
1=1 


Now 


,{y> 


V  l  z\yi  €  log  A,  log  (2A) ] )  =  Z  [  f  (x 

j-1  J  n=l  '{S1*n,C=l}  0 

1  (1/ (2A) )  Z  [  f  (x  x  )dx. 

n=l  J{S  =n,C=l}  y  1  n  1 


1 1  •  •  • i x  ) dx, 
i  n  1 


•dx 


(l/(2A))P(y)  (  Z  Z(/}  e  [log  A,  log  (2A)]) 
j  =  l  J 


As  A  -►  00 ,  lim  sup  <  <»,  and,  by  the  renewal  theorem 

S 


(v)  ^  f  vl 

lim  inf  Pvy'  (  Z  Z\y!  e  [log  A,  log  (2A) J )  >  0  , 
j-1  J 


from  which  Theorem  l(i)  now  follows. 


To  prove  the  second  part  of  Theorem  2(i),  choose  *n  t*ie 

interior  of  Q  such  that  F([(d.  ,6)-})  >  0.  Without  loss  of  generality 

1  2  n  (a)  } 

assume  that  >  0.  Denote:  F^  *  0,  *  mintnjz^p  +i  ® 


r*2 

or  ] 

exp{E”_  Z 

H 

J  i-1 

dF(y)  > 

A> .  Clearly  N1 

i 

(5) 

F 

Eo\: 

N 

t 

Now 

(6) 

Eori  i  Eo  1 

,  Y  =  min{ i | 


r„n  {y} 

cxp  £j-r,  ,+i  zs 

J  i-1  J 


r  «  E.N*  <  E  F .  E  Y 
t  U  A  0  1  0 


n  (to_ } 

V  J- 


j=l 


j 


In  a  manner  similar  to  Theorem  1  of  Poliak  (1983)  one  can  show  that 


<o 


(7)  AP  (Y-l)  — > 

'  A+°° 


2  ,  .  n  (o).} 

(1/Cj)pjy;  (  E  Z  1  >0,  n=l,2, . . .)dF(y)  . 

a)p  i=l 


Therefore  for  given  A^  there  exists  a  constant  Cp  such  that  if  A  A^ 
then 


(8) 


VY“1}  -  E0ri/(ACF  > 


Note  that 

(9)  EqY  -  1/P0(Y-1)  . 

Now  (5),  (6),  (8),  and  (9)  complete  the  proof  of  Theorem  2(i). 

PROOF  OF  THEOREM  l(ii),  THEOREM  2(ii).  Let  A  >  C  >  0  be  fixed. 

A 

Define:  ■  0.  For  j-1,2,...,  define: 
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Lj  *  mini 


r“_._  z!y} 


r  n  i..  .  1/ 

n|n  >  L,  ,,  E  e  1  *  1  dP(y)  >  A- 

J  i“LJ-l+1  CA 


*J  "  Li  ~  LJ-1 


j,n 


I 


i-L.+l  Zi  Lj 
6  J  Z 


l  i 


z\  z{y] 

e  dF(y) 


SLJ 

e  1"k  1  dF(y) 


H  *  min(n|n>_  L . ,  V.  >  A) 
j  j  J*n 

■  «°  if  no  such  n  exists 


VV  Vi 


J  -  min(j  |v.  „  >  A) 
J’Mj  “ 


if  n>  L. 


if  n-  L. 


NI"MJ 


R  -  ( 

J.n  ) 


Zn  z{y} 
i«k  i  N 

e  dF(y) 


for  n  >  L 


'J-l 


dFjty) 


h.  r  j  <j{y}  /  .  1*4  r  J 

-  I  >k  1  «<,)/  E  > 

W^l+l  /  J  k-Lj_j+l 


k  Zi 


df(s) 


5^j  ■  probability  measure  with  unit  mass  at  {y} 
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I 


1(0)  *  indicator  function  of  the  set  0 

xT  ■  min{n|/  exp{  Z  Z^}dF(y)  A,  n  ^  1} 

A  i-1  1 

=  ®  if  no  such  n  exists 

*1  -  *VTI <  “> 

K(y)  =  1/Cq 

Gp  =  /  K(y)dF(y)  . 

By  Theorem  3  of  Poliak  (1983),  ij  -*•  GF  as  A  -*■ 

Until  further  notice,  we  will  assume  that  the  support  of  F  is 
contained  in  a  compact  interval  [a,b],  0  <  a  <  b  <  00 ,  1(b)  <  00 . 

LEMMA  1.  For  arbitrary  0  <  U  <  1,  and  arbitrary  probability  measure 

<t>  whose  support  is  contained  in  [a,b],  0  <  a  <  b  <  00 ,  there  exists 

Bg  >  0  independent  of  <f>  such  that  if  B  >_  B^  then 

1^ 

(10)  1-T]  <  7^  <  1-Hl  . 

G* 

PROOF.  This  is  the  content  of  Theorem  1  of  Poliak  (1983). 


LEMMA  2.  For  arbitrary  0  <  n  <  1,  0  <  e  <  1  there  exists 

A^  *  A^(u,e)  and  C  *  C(n,e)  such  that  if  A  A^  and  one  chooses 


C.  *  C,  then 

A 


WHV 


u’ <  o. ». .  A  el*n 

Fj 


>  1  -  e  . 
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PROOP. 


(ii)  Pn(H,<»|a  )  -  ■ 

0  3  F, 

T  J 


if  R,  T  >  A 

3,ti  ~ 


;1/»J  v  ‘j.I.j74  “  *1.^  <  A 


In  a  manner  analogous  to  Theorem  1  of  Poliak  (1983b) ,  replacing 

OJ^.G^  by  a,b  respectively  in  (7),  one  gets  the  confergence  in  (7) 

to  be  uniform  in  measures  F  whose  support  is  contained  in  [a,b]. 

Therefore,  the  constant  C_  in  Theorem  2(i)  can  be  replaced  by  a 

FJ 

constant  C(Aq)  which  is  independent  of  (it  is  only  dependent  on 
a,b).  Hence  for  A  >  0 

Bo  r3,l  C(A  ) 

>m/ca>  1-4*7^  i -5s-  • 


Choosing  A  to  be  large  enough.  Lemma  1  in  conjunction  with  (11) 
complete  the  proof  of  Lemma  2. 


LEMMA  3.  For  any  e*  >  0  there  exists  6  >  0  such  that  if  one  chooses 
*  C  and  if  A  ^  C,  then 

E.  R.  ;  R,  >  6  <  e*  . 

0  J,Lj  j’Lj  caJ"  CA 


PROOF.  Let  X  be  distributed  as  under  Pq. 


L  »  ^  I  ^  5  fr 
J’LJ  CA 


±Eo 

;bebx 

a 

L 

S'1 

1+  z 
k-li-l+1 


e 


z^zW 

Li»k  zi 


dF(y) ; 


/bebX 

a 


lrl 

1+  Z  e  1  k  1 
k-lJ-l+1 


dF(y)  >  6 


•••“NY  4] 


"  1+ca/a  * 


This  can  be  made  to  be  less  than  £*A/C^  by  choosing  6  to  be  large 
enough . 


LEMMA  4.  Let  U  ~  \J(0,1)  be  independent  of  X^X^...  .  For  t  >  0  let 

”  >  Y"I>  Y«>  are 


,A 


(Ri  i  >  Yr(1!)  u  (R.  T 
j’Lj  G’A  J*Lj 


defined  by  pq(Qe>a)  *  e*  T*1®11  f°r  ^  >  0  there  exists  an  e  >  0  such 

that  E0(RjL  ;  Qe>A)/E0RjtL  <  X  uniformly  for  all  A,CA  such  that 
J  j 


A  >  C. 


PROOF.  Choose  e*  <  X.  Let  6  be  as  in  Lemma  3.  Let  e  >  0  satisfy 
e6  +  e*  <  A.  Then 

VYV  ■  eo[rj,Lj’  (("i.Lj  < s  t)  u  (s 


<  E.s  A+E.JL<  X  Ar  XEoR)  . 

AAA  j 
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t 


1 


LEtMA  5.  For  arbitrary  0  <  n  <  1  thare  exist  Aj  -  A2(n)  and  C  -  C(n) 
such  that  if  A  ^  Aj  and  one  chooses  CA  -  C,  then 

G_  Gp 


(i-n) 


<  p„(h.  <  »)  <  (l+n) 


J7i^  ' 


PROOF.  Choose  0<  a  <  rv  By  Leesu  4,  one  can  choose  e  >  0  such  that 
whenever  A  >  C. 


(12) 


Wj-  V*> 


Vo  *j.i 


<  n-a  , 


where  0  .  is  as  defined  in 

e ,  A 


4.  By  Leona  2,  there  exist  A^  and  C 
such  that  if  A  >  aax(A^,C)  and  one  chooses  CA  ■  C,  then 


Ke  * 


F0<Hj  <  -»»L  > 

^  <  C„  R.  TA  < 

Fj  J,Lj 


14a  i 


has  a  P0-probabillty  P0(K£)  I  l-€.  Note  that  since  P0(QeA)  >  Po((KE)C)’ 
for  any  set  S  £  (K£)c  n  ((^  A)c  there  exists  a  set  S  £  Qe>x  n  Ke  such 
^(S)  Pq(S*).  Obviously  R^  ^  on  S  is  larger  than  R^  L  on  S, 

J  * 


that  P 
and  therefore 


(13) 


E0(,J.Lj!  <KC)C)  \.A>  ■ 


Also  note  that  because  of  the  eart Ingale  property  of 


JpL^+1  •*P{Zi-k  Y  }-<n- Lj-1)  uadar  P0(*iV“  f°r  n>  LJ-1)* 


it  follows  that 


'o  s, 


Eo  “j.ij  '  E«dYr> 


Lj  {y} 

Ert  /  K(y)  E  exp'  Z  z\yJ  dF(y) 

0  k"Lj_l+1  [i"k 


-  E0<Lj-Lj_i^  f  KCy)dF(y) 


E0  Rj,Lj  GF  * 


Therefore,  by  (12),  (13),  and  (1A) , 


?0<»3  <  ->  -  »o(,0OIJ<*1^):  Ke  °  <Ke)C) 

<  (lt<.)Eo»j,Lj  +  Eo['j.L3/A;  <VC) 

<  tt+oJE,  Gp/A  +  Qc  A) 

<  <l  +  o)E0Rji,  Gp/A  +  V* 


(1+n>  A/E„  R 


0  j,L. 


Likewise, 


p0(b3<”)  i  WV*1^’1  *E> 

>  (l-a)E„«)>L  =FjM;  V 

>  (1- t>)E0(R3i^  Gp^/A)  -  Eo(EJ,Lj/A!  (V  ] 

>  (1  -  cOEqRj  ,  G,/A  -  *o(EJ.E3/A!  Oc,A» 


I 


1 


>  <l-a>E0R  Op/A  -  (T)-a)E0R  V* 
J  J 

Gp 

“  (1  -  D)  T7v  r  • 

Rj,L4 


LEMMA  6.  For  arbitrary  n  >  0  there  exist  A^  ■  A^Cn)  and  C  *  C(n) 
such  that  if  A  A^  and  one  chooses  -  C,  then 


l  -  n  < 


wjv*0 

— vk - <1  +  n  • 


PROOF.  Note  that  e0(Vj>h  »  H j  <  °°>  "  E0Rj,L  and  ®° 

„  |H.  <  »)  -  En  R,  T  /Pn(H.  <  «) .  An  application  of  Lemma  5 

0  j  J  V  J  ft  i-j  V  J 

completes  the  proof  of  Lemma  6. 

LEMMA  7.  For  arbitrary  0  <  n  <  1  there  exist  A4  «  AA(n)  and  C  -  C(o) 
such  that  if  A  ^  A^  and  one  chooses  CA  -  C,  then 


E0<VJ,M  lVJ,M  i»> 

1~T'< -  a/g!  - <1+n  * 


PROOF.  Let  A  be  large  enough. 


(15)  E. 


VYh/  V1.H 


a»ri»,  E0<vJ1(j;  V}  Bj>a)  <  e0(vJ  Mj;  h^-), 

<  PqWj  <  «) ,  and  PqOTj  >  A)  -  P(J(Hj  <  -)  -  PQ(Lj+1  <  ^  <  -)  .  Denote 
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Qr(*)  -  P.(y)(*)dG(y).  For  x  >  0 

v>  a  i 


P0(x  +  Lj<HJ<“)  "  E0P0(x+Lj<Hj<”l3Lj) 

-  o(l)  E0  Rj  L  /A  -  o(l)  P0(Hj<»)  , 

where  o(l)  ♦  0  as  x  +  «  uniformly  in  3^  ,  A  for  fixed  CA  =  C. 

Also,  PQ(Lj+1<  Lj  +  x)  -  PqO^Ix)  =  ZXml  PQ(Rn>  A/Ca)<x2  Ca/A.  So, 


for  x  >  0 


+  po(Lj+l-H3-x> 


<  2P0(x<nj<-)  +r0CL)+1<  ,) 


<  0(1)  Pq(Hj  <  -)  +  *  Ca/A  . 

Since  (by  Lemma  5  and  Theorem  2(i))  Pg(Hj <  °°)  i9  of  the  order  of 
magnitude  of  1/CA>  choosing  A,  CA  large  enough  will  cause 

Po^+^V^/PoV")  t0  be  arbltrarily  s®®11*  i*e., 

Pft(V4  u  >  A)/Pn(H. <  °°)  to  be  arbitrarily  close  to  1.  Similarly, 

0  j*Y  0  j 


V^.Mj5  VJ,Mj<A*  Hj<W>  "  E0  QFj  TA/Rj  >L^  >  N1  ^j.Lj 


-  o(l)  Pq(H  <•)  •  A  , 


I 


where  o(l)  +  0  as  A  4  Therefore,  choosing  large  and  very  large 
A  one  can  get  Eq(Vj  M  »  M  ^  A)/Eq(Vj  R  ;  H  ^  <  “>)  to  be  arbitrarily 
close  to  1.  Hence,  one  can  make 


(16) 


V,-*'  /  Eo<vi  .a, :  »j<*> 

Wmj  ±A>  /  W0 


be  arbitrarily  close  to  1. 

6. 


7  now  follows  from  (15) ,  (16) ,  and 


LEMMA  8.  For  arbitrary  0  <  n  <  1  there  exist  A^  -  A^(n)  and  C  *  C(n) 

such  that  if  A  >  Ac  and  one  chooses  C.  •  C,  then 
—  j  A 


1  «  T)  < 


E0VJ,m, 


"a7g: 


<  l  +  n  . 


PROOF.  Denote  V _  u  -  V  .  „  -  0. 


(17)  E0VJ,Mj  "  fJ~j  Vj  ,Mj  dp0 


00 

■  E 

j-1 


vl-«3  dp° 


7i,M1<A’  Vj  ,m^  - 


'  A  ]  v^mj  *° 

V 


. J-2iVJMj>A) 
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I 


] 


(20)  £  P0(»1>Mi<».i-l,0 . J-2!  Vj  Hj>A) 

<  2^  VVi,H  <A.  1--1.0 . j-3)  PgO^  V2  M  >A) 


■  U  +  V’Wm^*'  V2,M2i‘>  • 


Denote:  J  . .  ■  min(n|n  odd,  V  u  >A},  J _ *  min{nln  even,  V  „  >  A)  . 

odd  n,M  even  n^M  — 


Since  J  *  mln{j  ,.,J  }  <  J  ..  +  J  , 

odd*  even  —  odd  even* 


(21)  EqJ  <  EQJodd  +  EoJeven  -  Pn(V  >  A)  * 

U  — 

Therefore,  because  of  (17)-(21),  It  only  remains  to  show  that 
P0(Vi  M  V2  1  A)  can  be  made  to  be  sufficiently  small. 

<22>  V2,«2i«  ‘  VVl,M1ift'  V2.KjiW 


+ B2,M1iCA  ’  V2,M2-A) 


2(i). 


Suppose  that  A/Ca  ^  where  Aq  is  a  constant,  as  in  Theorem 

Note  that  on  (R,  „  <  A/C.} 

Z.M^  A 

^,1^  >  A) 


f  **1  l‘Z  Z{y}  L2  l1 
■J  E  e  **  1  +  E  e 

u(/(k-L1+l  k-M^l 


I 


i  P0(V2,M2>  AlVl>Ml>  A*  R2,M1<^>P0(V1,M1>  A) 


±Eo 


“2.1 


V1,M  -A*  R2,M  <  C. 
1  1  A 


P0<Vl,MliA) 


1  +  C(A  ) 

<  - p  (V  >  A)  . 

CA  0  l.Mj- 


Now  for  any  x  >  0 


(24)  P0<v1>Mi!A,  r2,m  -C~*  V2,M  -A) 

-po(vi,m.-a’  R2fM.i£»Mi±Li  +  x) 

1  1  A 

+  P0(V1,M1-A’  R2,M.-^*  M1>L1  +  x) 

1  1  A 

<  V  M1<  Lx  +  x)  +  P0(Vl  M  >  A,  M1>  Lx  +  x) 

x  A  1 


(25)  p0(R2  mi<li  +  x)  1  ^  P0<Rn-F'>  -  a7c“  » 

’  1  A  n»l  A  A 
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I 


and  as  in  the  proof  of  Lemma  7 


(26)  P0(V1>M  >  A,  Mx  >  Lj+x)  <  P0(Ll+x  <  <  «)  «  o(l)  PgO^  <  ~) 

where  o(l)  ■*  0  as  x  -*■  00  uniformly  in  J  ,  large  A,  for  fixed  C.  -  C. 

lj  * 

Now  (17)-(26)  in  conjunction  with  Lemma  5  and  Lemma  7  and  its  proof 
complete  the  proof  of  Lemma  8. 


LEMMA  9.  Let  X  >  0.  There  exist  C  =  C(X)  and  A,  =  A, (X)  such  that 

o  o 

if  A  >  A,  and  one  chooses  C,  =  C  then 
—  o  A 


M, 


M, 


(27) 


E  /  Z  exp{  Z  Z.y  }dF(y)  <  XA  . 
k=Lj+l  i=k  1 


The  sum  in  (27)  is  understood  to  be  zero  if  M  =  L  . 

J  J 


PROOF.  It  is  enough  to  prove  that  under  the  conditions  described 


(28) 


for  then 


E0<R2,M1IV1,M1  -  A)  -  XA  ’ 


M, 


M. 

J  J  f  v'l 

E  /  Z  exp(  Z  Ziy;}dF(y) 

k-Lj+1  i=k  1 

MJ  MJ 

Jodd  odd  <•  i 

E „  /  Z  exp{  Z  Z  }dF(y) 

k=L  +1  i=k 

Jodd 


+  E„ 


MJ  MJ 

even  even 

Z  ex; 


k=Lj  +1  i-k 

Jeven 


even  r  -i 

p{  Z  Zy)}dF(y) 
* 


<  2XA 
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where  J  ,,  and  J  are  as  in  the  proof  of  Lemma  8. 
odd  even 

On  (A/C.  <  R.  ,  <  A) 

A  —  1»L^ 


(29) 


Eo(R2,«y  V1,H1  i A>  ■  H1  i  V  • 


Let  x  >  1.  Note  that  <_  Lj+x  <  u  <_  L2  £  Lj+x} 

u  {Lj+x  c  <  L2}  u  {Lj+x  <  =  L2) .  We  will  analyze  the  expectation 

in  (29)  on  each  of  these  four  events  separately.  Note  that  R2  H  <  A^CA 


1*2,1^  °"  IH1  '  L2K 


<30>  VR2,y  “i  i  Ri+X  •=  l2>  i  Eo<R2.y  H1  i  V0  -  Eo(R2.yx>  ■  * 

<31>  Eo^R2,H1'  H1  i  L2  i  LltX>  i  E0<R2,L2:  H1  i  L2  i  V*'  i  E0(R2,L1+*> 

(32)  E0(R2H^;  Lj+x  <  W1  <  L2)  <  (A/Ca)  P0(L1+x  <  Hx  <  -)  . 

(Later  we  will  let  x  be  large  and  will  evaluate  (32)  as  in  the  proof 

of  Lemma  7.)  Denote:  x  =  ^ (L^+x)  v  (k-1)  <  H^-L^ .  Given 


^*1  ’^1’  * '  *  *^l 


1  ,{y>- 


(33)  Eq(R2 ^  yx  <  Hj  -  L2)  -  E0  /  expt^  Z’  }l(EkjX>dF(,) 


1 

(34)  Eq  f  exp{^  Zly)}l(5k,x)dF(y) 


r(y) 


k-1 

Z 

ziy)> 

H1 

exp{  Z 

i=L  +1 

1 

i=LL+l 

exp{- 

k-1 

Z 

z!y,)KHk 

i*=L  +1 

1 
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'  <£1  <Mpt-  *1  z[y,,1<5».x>lxL1+r---Vi,dr<5') 

1  'l’+I  “pf"  lm^+1  Ziy  ,PLyil(=k.xlXL1+l . X|t_1)dF(y) 

’  Eo  . \-l>dF«  • 


Let  \J)(a)  >  h  >  0.  Let  Eg  >  0,  jg  *  (log  A) 


For  large  enough  A 
L  +j-l 

’  \r  v  /  i  1  \  4  /t 


there  exists  e.  >  0  such  that  for  all  j  >  j_  P-(Z  T  4.1  >  (j-l)h/b) 

L  +j-l  0  0  imLl  1 

<  expC-e^j}.  On  (Zi=L  +1  <_  (j-l)h/b):  for  n  >_  j-1  +  L 

Vx  n  <_  exp{j  (h-ip(a)}  f  exp{yZ"ML  +j  -  (n-j-Lj+l^y^dF^y)!^  L  .  Let 

"  •  s+J-  ut  Bi,k  ■  "‘"{"i"  i k-  vi,„i  *>•  <y!i<«i,k  - k) 

^  (j-1) (<p(a)  -  h)/ij;(y)b).  So,  for  large  enough  A,  for  a  <_  y  <_  b,  there 

exists  e2  >  0  such  that  for  all  j  jQ  k  “  k  <  j-1)  OJKa)  - 

h)/(^’(b)b)  <_  exp{-e2j}  for  all  a  <_  y  <_  b  (see  Poliak  and  Slegmund  (1975) 

for  an  example  of  the  considerations  Involved),  Since 
b  n 

L_  <  mln(nj/  exp(y  Z  X  -  (n-k_l)i(;(y)}dF(y)  >  A/C  } ,  there  exists 
1  a  i-k  ,  , 

e2  >  0  such  that  for  all  j  ^  jg  Pl  +l^L2_lt  >  ('Ka)^)/^1  (b)b) 

<_  exp{-e3J)  for  all  a  <  y  <  b  if  A  is  large  enough.  Therefore,  for 
j  ^  jQ,  k  *  Li+J*  if  A  is  lar8e  enough. 


E0  r;yii  <Ek,  A1+1 . W 

i  E0  P^il  <(k-l>  v  <Lrt)  <  l2  •  . Ek-1> 

<_  exp{-£jj)  +  exp(-e2j)  +  expt-e^j} 


and  so 


» 


SB 

(36)  Eq(R2  h  :  Lj+x  <  Hj-Lj)  £  Jo  +  1  +  exP^"e2^  + 

1  j"J0 

^•"^0 

By  letting  x  be  large  enough  -  such  as  (log  A)  -  one  gets  by  virtue 
of  (30)  -  (36)  that  EA(R.  M  ;  V,  u  >  A) /A  is  arbitrarily  small  for 
large  enough  A,  from  which  (28)  follows. 


LEMMA  10.  Let  X  >  0.  There  exists  C  -  C(A)  and  A?  -  A?(X)  such  that 


if  A  >  A,  and  one  chooses  C.  *  C,  then 
—  /  A 


0  <  ER  .  -  EnV.  <  XA  . 
-  <  0  J-Mj  - 


PROOF.  Clearly, 

M  MJ  2{y} 

j  j  J  *i  j_l 

(37)  V  "  VT  M  +  1  e±mk  dF(y)  +  E  V1H 

**A  J’MJ  k-LyU  j-1 


Therefore,  by  Lemma  9,  it  suffices  to  show  that  eq  ^j“l  Vj  M  <  ^  ^ 

J 

for  appropriately  chosen  C.  Let  J^^  and  Jeven  be  as  in  the  proof  of 


J-1 

(38)  En  Z  V 


0  2  V  -  Z  E  (V  J  j  <  J  -  1) 

°j-l  j,MJ  j-1  °  j,MJ 

00 

-  1  Eo(vi  M  ;  j  £  J  -  x> 

j-i  0  j’Mj 


E0  Z  Vi  M 
°j-l  j’Mj 


J-1  J-1 

E0  1  M  +  E0  2  V1  M 
°  j-1  J,MJ  °  J-2  j,Mj 


1 


•’odd-1 

"even”1 

<  E  V_,  w 

+  E.  E  V.  M 

0  j-1  j*Mj 

0  S-2  J>MJ 

^odd 

^even 

J  ..-2 

J  -2 

odd 

even 

-  E.  I  V,  w 

+  E-  E  V, 

0  J-1  j,MJ 

0  J-2  J‘MJ 

^odd 

^even 

-  ®0(vi,m1!vi,m1 

<  A>VJodd-2  +  Jeven-2> 

1  VYmJYm, 

<  A)(E0  Jodd  +  E0  Jeven> 

<  EA(V,  M  |v.  „ 

"  A)  ^i.m,  TV  ■ 

Now 


(3,»  e0(vi,m1-  Vl,«1  <  «  -  *0  vi.m1-eo<vi,h1=  V”> 


+  E0<V1,H1'  H1<">-E0(V1,M1'  ''l.Mji** 


-  E0Rl,l1-E0El,L1  +  E0(Vl.H1i  V> 


1  - 


E0(T1,M1‘ 


eo<vi,h1'  Hl<“> 


°W  • 
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I 


where  o(l)  -*•  0  as  A  •  as  in  the  proof  of  Lemma  7.  Since  (as  in  the 


proof  of  Leoma  7)  tyv1>Mi>  «>  +  1  as  A  +  •,  (38)  and  (39) 

with  Theorem  2(i)  and  Lemma  5  complete  the  proof  of  Lemma  10. 

PROOF  OF  THEOREM  2(ii).  Since  (see  (37))  R^*  >  Vj  M  >  A,  it  follows 

A  *  J 

that  N*  and  so 

(40)  E0^<E08J-E0SJ.  . 

Denote  J*  .  next )  |  end  ^  <  A,  or  Lj  . 

V 

f  J  ^  z{yJ  J*-l 

RF  *  V  _  +  Z  e  i-k  1  dF(y)  +  Z  V  . 

NA  J*,NA  •i“1  j^A 

Since  V.  „  <  A  for  j  <  J*-l  and  since 
J’Mj 

E0(r]-il  v(  „F<  ■  h<*£?  VJ ,N*’  "M*-  “  follo“8  (sl"“ 

J,na  a 

Mt  -  N?)  that 


J  A 


J*-l  J-l 

E  Z  V  <  E  Z  V  , 

0  j-i  °j-i  3iMj 


which  in  turn  is  bounded  as  in  Lemma  10  (see  (38),  (39)  above)  by 


A  o(l). 


<  r"*  M 

fair  4 

Z  e  1  K  1  dF(y) 
k-Lj*+! 


f  Nf  r  1 

N.  £  A  viyj 

A  ^i-k  4i 

En  Z  e  1  *  1  dF(y) ; 

Jk-L^l 


F  * 
NA<NA 


i 


I 


choosing  C  to  be  large  enough  X  would  be  arbitrarily  small,  and 
P.(R  -  V  >  AA)  <_  e  for  all  large  enough  A.  I.e.,  for  large 

enough  A, 


e£P„(V  .  5  R  ,  -  AA)  >  P  (V  <(1-A)A) 

°  <  0  j\< 


Let  N**_^A  denote  N*^  ^  A  when  one  chooses  *  (l-X)C^. 

It  follows  that  for  large  enough  A,  with  CA  ■  C  as  above. 


Vu-AM  i  <>  i  1  -  E 


Therefore,  if  C  was  chosen  to  be  large  enough. 


<41)  Va  ■  Vv  +  Eo(Nv  ’‘I'V-aia* 

itlt  V  AA  p  V  AA 

^E0(N(1-A)A;  +  W  VN<1-X)A> 


E0N(1-A)A  "  E0(N(1-A)A"  na;  VN(1-X)A> 


-  E0N(1-A)A  "  e[E0Rl,L1  +  E0N(l-A)A] 


** 


>  (1-  2e)EQN(1_AjA  , 


for  all  large  enough  A.  Since  e  and  A  can  be  arbitrarily  small,  the 

fact  that  E0N(1-A)A  *  EQRE**  coupled  with  (40),  (41),  Lemma  10, 

(1— A) A 

and  Lemma  8  complete  the  proof  of  Theorem  2(ii)  for  the  case  where 

the  support  of  F  is  contained  in  [a,b],  0  <  a  <  b  <  00 . 

If  (0,“)  C  ft,  a  »  0,  and/or  b  -  00 :  If  one  replaces  dF  by 
F*  F* 

dF*  -  1(— ,n) dF,  then  NE  £  N  n  (letting  N  n  have  the  obvious  meaning, 
n  n  a  a  a 
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despite  F*  not  being  a  probability  distribution)  and  so 

Vi  ^  A  V  +  o(1))’  o(1)  -  0  as  A  -  and  C*  is  the  constant 

in  Theorem  2<ii)  (as  described  after  the  statement  of  the  theorem). 
For  arbitrary  a  >  0  define  Fq  a  “  (l  +  ot)Fn. 


,,  <  A  z(v>  r»I  i*  i'1 

(I/O  A  Zi  dP(y)  +  r  £  e 

-0  i  k.i  l  k.1 


dF(y) 


E„N?  |1/n<iF(y)  +  £°  dF(y)  . 


Therefore, 


i  /  N*  r^A  7^  rco  ***  r^A  7{y> 

fl/n  A  E.  .  2/  r®  A  E  2/ 

Pn  E  e  1  K  1  dF(y)  +  E  e  1  K  1  dF(y)  >  AA 

°  J0  k-1  Jn  k-1 


'0  k-1 


r  dp<y>  +  r  dp<,) 

A  A 


which,  for  any  A  >  0,  can  be  made  arbitrarily  small  by  taking  n  to  be 
large  enough.  Now  for  A  sufficiently  small 


PO£<NA"’a) 


,1/n  <  Z.\  Z.f,! 


NI <  na  ’  J  E  e  1  V  1  dF(y) 

b  k-1 


_  »f  X  M 

+  f  E  e  i_k  1  dF(y)>AA 

b  k-1 


[,1/n  /A  (y) 

„  n  f  '  v  i«k  Zi  . 
<  Pn  E  e  dF(y) 

v  J0  k-1 


t 


r  ^  ziik  ziy} 

E  e  1  *  dF(y)>XA 

4  k«l 


In  other  words,  for  arbitrary  e  >  0,  Pn(N^  >  N  n,0t)  >  1  -  e  for  large 

0  A  A 

I  F 

enough  n  and  A.  It  is  easy  to  see  that  EgO^”'*1  ~  N^j N^n,0t  N^) 

F  F 

i  Va"'"'  Ifenc*  vli  V*n'“<1-£)- 


Letting  e  -*■  0,  a  -*•  0  completes  the  proof  of  Theorem  2(ii)  for 
(a.b)  C  (0  «). 

If  w  =  sup{y|y  €  fi}  <  00  and  I(y)  +  “  as  y  +  w,  a  similar 
proof  is  valid,  letting  b  approach  w  instead  of  °°. 

The  proof  for  (a.b)  £  is  similar. 


PROOF  OF  THEOREM  l(ii) .  The  proof  of  Theorem  2(ii)  can  easily  be 
adjusted  to  be  a  proof  of  Theorem  l(ii).  In  the  general  non¬ 
arithmetic  case.  Stone's  (1965)  results  can  be  replaced  by  the  stand¬ 
ard  renewal  theorem.  (There  is  no  need  for  uniformity  of  the 
renewal-theoretic  results  as  the  support  of  the  mixing  measure 
F  ■  *5(0}  in  this  case  is  made  up  of  one  point.)  The  details  are 
omitted . 
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IV.  THE  AVERAGE  RUN  LENGTH  WHEN  v  *  1 


Define 


-y.e  _ 


eJ0)  log 


-  -  z.k.  zjy) 

1  +  E  e  1-1  1 

k=l 


~y,0 


lim  E 
B-k» 


(6) 


MB 

i=l 


Z  z[y}-  B 


ry,9  _  ry,e  ry»0 

1  "  3  “  2 


C®,F  =  -  |  log  [2tt(F'(0))2/^,(0)] 


C®  =  \  log  1(0)  -  \ 


0,F  0,F  0,0  0,0  0 

C1  '  C2  +  C3  "  C2  '  C4 


y  0 

The  computation  of  C^’  is  an  application  of  renewal  theory.  The 

v  0 

calculation  of  C^’  seems  to  be  feasible  only  with  the  aid  of  Monte 
Carlo. 


(A) 

THEOREM  3.  If  y,0  e  ft,  0  <  y  ip' (d)  -  ip(y)  <  and  the  -distribu¬ 
tion  of  log(f  (X1)/fQ(X1))  is  non-lattice,  then 


f(0>  M  - 


y\J>'  (0)  -  iHy) 


[log  A  +  C^’6  +  o(l)] 


where  o(l)  +  0  as  A  ■+■  00 . 
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THEOREM  4.  Suppose  F' (y)  =  dF(y)/dy  exists,  is  positive,  and  is 
continuous  in  an  open  neighborhood  of  9  e  ft.  Then 

E<0)  <  =  “y  [log  A  +  4  log  log  A  +  c0,F  +  o(l)] 
where  o(l)  +  0  as  A  + 


PROOF  OF  THEOREM  4,  THEOREM  3.  For  the  proof  of  Theorem  4,  assume 
(without  loss  of  generality)  that  0  >  0.  Consider  first  the  case  where 
F  is  concentrated  on  [0^,0^]  where  0<9q<9<6^<°°  are  such  that 
yxp *  (0)  -  ip(y)  >  0  for  8q  y  <  0-^  and  F  has  a  derivative  F'  which  is 
positive  and  continuous  on  For  Qq  —  y  —  0i  denote 

wn,y  =  1+  Eeili 
k=2 


Note  that  Wn,y  converges  a.s.  p50^  as  n  -*■  00  to  a  random  variable  W  .. 

l  y,0 

(9) 


Since  E  (VJn+F,y  -  Wn,y)  =  E  exp{-y  E.n  X.-n  <Ky)  ]  F 

-  n=m  K  }  i=l  i 


n=m 


m->-  <» 


>  0 


(0) 


uniformly  in  y  c  [0,0,],  it  follows  that  W  is  a.s.  P 
U  i  y,t)  1 

a.s.  p(0> 

continuous  in  y  6  [0^,0^],  and  Wn,y  — n  ~ — *•  q  uniformly  in 

y  €  [0,0  ] .  Note  that 

9  En  7fy) 


„  r<3 ,  L.  ,  L- 

-  J  1  e  1=1  1  Wn,y  dF(y)  . 


The  proof  of  Theorem  4  now  follows  the  proof  of  the  asymptotic 
formula  for  the  expected  sample  size  of  power  one  tests,  based  on 
non-linear  renewal  theory  (cf.  Lai,  Siegmund  (1977)).  The  details 
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presented  here  follow  the  proof  presented  In  Woodroofe  (1982) 

Section  6.3.  With  minor  modifications,  the  proof  is  the  same. 

One  difference  is  that  Woodroofe's  u  (Y  )  now  has  ir(ds) 

n  n 

replaced  by  Wn,s  Tr(ds).  Note  that  the  upper  bound  on  the  newly 

defined  u  ( Y  )  is  not  uniform  in  Wn,S,  One  must  show  that  (13)  and 
n  n 

(14)  of  Section  4  of  Woodroofe  (1982)  are  nevertheless  satisfied. 
One  can  dispense  with  (14)  by  noting  that  Wn>  _>  1  .  To  show  that 
(13)  is  satisfied,  it  more  than  suffices  to  prove  the  existence  of 
a  constant  a  >  0  such  that 


Let  e  >  0,  A  =  min{n|  |x  -  (©> }  £  £  for  a11  min).  Suppose  that 

e  is  small  enough  so  that  there  exists  6  >  0  such  that  I  z!^  Sn 


if  n  >  A  for  all  0Q  <  y  <  9^  There  exists  y  >  0  such  that  U'(0-y)  + 

(y)  _  \Jj(0)  |  <  y  for  all  0O  <_  y  <_  9^  There  exists  a  constant  5  >  0 
such  that  P^(A=X)  £  exp{-<5X)  .  Choose  1  >  a  >  0  such  that  ay  -  6(l-a)  < 


Now 
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~k-l  ly) 

(9)  f  J  e-Ei=lZi  dF(y)|A,x  < 
1  j/k-2  1 


,a  \  5-k-l  Jy) 

(6)  el  l  e  El-1  Z1  ( 

1  'A  Vm') 


dF(y) 


90  k"2 


P<9>(A-A) 


_ I _  fl  \ 

?{0)(A  =  A)  'e0  k« 


I  e[*(0-y)-hP(y)HK6)](k-i)dp(y) 

k-2 


1  1  yA 

-pJ9)(A-x) 


By  Jensen's  inequality, 


Ef ’  f 1  ”y,8<1F(E>  01  ‘  Ef '  Ef'  f 1  “y  6dF<E>  “  A 

e0  [( eo 

-  “  1  1  yA  2  -  e“e|a  (0) ,  . 

A-l  [Pf)(A=A)Y  1  -e-e)  1 

The  inequality  (42)  now  follows  because 

E  [~70T^ - 7  eYX)a  P,(9)(  A  -  A)  =  \  Z  eaYX(P(0)  (  A  -  A))] 

A=1  pJ0,(A  =  A)  Y  1  Y  A=1  L 


<  _1_  ”  eA(ay-(l-a)6) 


—  a  ,  , 
Y  A-l 


<  «  . 


To  complete  the  proof  of  Theorem  4  for  the  case  that  F  is  con¬ 
centrated  on  [0g,0j]  as  above,  one  need  only  show  that  (16)  of  Woodroofe 
(1982),  Section  4,  holds.  For  this,  following  Woodroofe’s  (1982  Section 
6.3)  proof,  it  suffices  to  note  that 


I 


which  is  equivalent  to  (16)  of  Woodroofe  (1982),  Section  4. 

For  the  general  proof  of  Theorem  4,  let  F  be  a  measure  on  the 

real  line.  There  exist  constants  0  <  £  <  l(0)/2,  co  >  0,  and 

O<0Q<0<0^<a°  such  that  yip' (6)  -  ip(  y)  >  0  for  y  c  t0Q»0j]> 

max{yijj'  (0-w)  -  <My),  y^'  (0+w)  -  Tp(y>)  <  £  for  y  j  and  F(y) 

has  a  derivative  F' (y)  for  0^  £  y  £  0^  which  is  positive  and  continuous 

for  —  y  —  ®l‘  Since  ^  1°8  A)/I(0))  is  arbitrarily  small 

(6)  Fi  F 

when  A  is  large  enough,  and  since  for  all  x  >  0  E  (N.|N  >  x)  <  x  + 

X  A  A 

(2  log  A)/I(0)  for  large  enough  A,  it  suffices  to  show  that 


(log  A)P}0>{ 

[n=l , . 


,  (2  !og  A)/l(9)  p_[6^i9^] 


Xi-(n-k+1)Ky)dF(v)  >  _3A_1 

*  e  dF(y)  >  lQ  A 


The  remainder  of  the  proof  is  therefore  an  analysis  of  this 
expression. 


39 


Let  -  00  <  0*  <  0  <  0^  <  0*  <  00  be  such  that 

-  S  =  max{y4/'  (0)  -  *Ky)  |y  (0*,0*)}  <  0  . 

In  the  same  manner  which  lead  to  (42)  above,  it  can  be  shown  that 
there  exists  a  constant  a  >  0  such  that 


=  v<0) 


r  =  e 


“  y^_-,  x.-kiKy) 

£  e  1-1  1  dF(y) 


R-ieJ.eJ] 


<  00 


and  hence  by  Jensen's  inequality 


(45)  (logA)pJ0)^ 


max 


n=l . (2  log  A) /I (9 )  jF-[0*,6*] 


n  y £.  .  X  -(n-k+l)i^(y) 

I  e  1_k  1  dF(y)  >  A 


k=l 


-  log  A 


(2  log  A)/I(9)  ,  .  n  y£.  .  X  -(n-k+lM>(y) 

<  (log  A)  £  ((/  £  e  1_k 

n=l  k=l 


>  ( — - — )a) 
-  Hog  A;  ' 


2  log  A  _ T_ 


I(9)  (A/log  A)a 


- ■>  0  . 

A  -*■  00 
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dF(y))“ 


I 


For  large  enough  A 


(46) 


n-1 . (2  log  A) /l  (6) 


n  yE^  Xj-(n-kfl)Wy) 
E  e 
k-1 


nl'k+i  *  xi  e  (*'(e-«).*,(0+«)) 


dF(y) 


£  max 

n-1, .... (2  log  A)/I(0) 


I  ? 

>  k-1 


eCCn-k+l)  dF(y) 


<  max  e _ 

n-1,.,., (2 log  A)/I(0)  l 


C(n+1) 


el  .25/1(0) 

TA 


< 

log  A  • 

Let  ri  >  0  be  such  that  X^/k  t  0|>'  (0  -  <o)  ,\Ji'  (0  +  ft>))) 

£  exp{-nk)  for  all  k.  Let  X  >  0  be  such  that  tE^^CX^  -  l|)' (0)) 

2 

£  Xk  for  all  k.  For  large  enough  A  and  for  n  £  (2  log  A)/I(9) 


p(0)(fet  5  yEi-k  xi_(n-k+l)'Ky) 
1  \  « e 


I  xt  «  »' (9 -<■>),*  •<«+<■>» 


log  a 
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»  •:  *i 

E  e 
k-1 


- i-T-  E  x,  t  fl>'(0 (8 +  <*>)) 

n-k+i  i,k  i 


log  A 


£  max  e 

|k— 1  •  •  • « •  ti 


°t  ^-i  xi 


i  e  x.  t  (^'(e-^.^ue  +  w)) 

*  i«i  1 


log  A 


<  E  P 

k-1 


(8) 


E  (X, -^’(0)) 
i-1 


£  E  X.  $  (ip'(0-w),*,(0  +  a))) 
|K  i-1  1 


>  _  log  A 
Q1 


— —  log  log  A  -  k^’(0)  +  log(I(0)/2)> 


J 


n  a(  log  A) 
<  E 
k-1 


1/4 


Xk 


•  log  A  — log  log  A  -  kr(6)  +  log(I(0)/2) 

A*  A* 


n 

E 


k-(log  A) 


?(B) 
1/4  F1 


±  E  X.  t  0|»'(e-a i),<T  <«  +  <*»>) 
11  1-1  1 


X(log  A) 


3/4 


^  log  A  -  -2-  log  log  A  -  (log  A) 3/4  t|>'(0)  +  log(I(0)/2) 

ft*  A" 


+  e-n(iog  a) 3/4  l 


1  -  e 


-n  * 
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It  follows  that 


(47)  (log  A)  P. 


(9) 


max 


ej  n  yEj_kX1-(n-k+l)iKy) 


(k-l,...,(2log  A)/I(6)  ^ 


f1  i 

k-1 


^-£71  Z  XA  fe  (4),(e-a»),^,(9  +  <o)) 


dF(y)  >  I5Fa 


A-x» 


0  . 


In  a  similar  fashion  one  gets  that 


(48) 


(log  A) 


(9) 


en  n  X  -(n-k+l)i|>(y) 


max 

lk«l,...,(2log  A)/I(0) 


ro  n  y\-kxi 

k-i 


n"IV+'l  E  x!*  <**<e -«>.♦' <e  +  «» 

i-k 


dF(y)  > 


log  A 


A'*°° 


0  . 


Formulas  (45)-(48)  account  for  (44)  and  so  the  proof  of  Theorem  4 
is  complete. 

The  proof  of  Theorem  3  follows  along  similar  lines.  The  details 
are  omitted. 
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V.  MONTE  CARLO 


A  Monte  Carlo  study  was  made  for  the  normal  model  with  unit 

variance.  Letting  f  denote  the  density  of  the  N(0,1)  distribution, 

0 

{0} 

simulations  of  N  ,  R  were  made  for  6  3  .4,  .8,  1.0,  1.2,  1.6, 

A  N(0} 

2.0,  2.5,  3.0,  4.0  and  A  *  10,  20,  30,  100  using  ~  N(0,1)  random 
numbers.  For  each  of  the  36  combinations  of  and  A,  10,000  realiza¬ 
tions  were  obtained.  The  results  show  the  asymptotic  formulae  (derived 

{0} 

in  the  previous  sections)  to  give  a  very  good  picture  of  Entr  even 

U  A 


for  surprisingly  low  values  of  A. 


(9)  „{0}, 


As  expected,  the  Monte  Carlo  estimate  of  Eq(r  (q}  “  NA  >  was 


zero:  in  only  one  of  the  36  cases  did  (R 


{0>  .Je) 


N 


N 

,{0}  „{0)i 


{0} 


N  )  exceed  two  (Monte 


Carlo)  standard  deviations  of  R  {g}“  N^  •  Results  of  Lai  and  Siegmund 

A 

(1977)  lead  one  to  conjecture  that  the  linear  correlation  coefficient 


.,{0} 


,{0} 


between  N  and  R  is  asymptotically  (A  -*■  °°)  zero.  The  Monte  Carlo 

N* 

A 

results  support  this  conjecture  -  the  highest  Monte  Carlo  correlation 

{0}  {0} 

between  and  R  was  .0234.  (In  28  of  the  36  cases  the  correlation 

N . 

{0}  A  r  0 1 

between  N  and  Rlr--i  was  not  significantly  different  from  zero  at  a 

NA 

5%  level  of  significance,  and  in  all  of  the  36  cases  this  correlation  was 


not  significantly  different  from  zero  at  a  1%  level  of  significance.) 

0 

Therefore,  estimates  of  EqN°  were  made  using  a  linear  combination 

{0}  {0} 

Ve  N»  +  <l  *  “*,e>  R  {0}’  W*'ere  aA  0  was  c^08en  to  minimize 

NA 

2  {0}  2  {0} 

a  Var  N^  +  (1-a)  Var  R1^  (the  variances  being  Monte  Carlo  vari- 

^A 

ances).  The  results  are  presented  in  Table  1. 


Values  of  E.N  predicted  by  asymptotic  theory  (TH)  and  estimated 

v  A 

by  Monte  Carlo  (MC) 


10 

20 

30 

100 

E  N{e) 

eona 

S.D. 
of  MC 

viel 

S.D. 
of  MC 

E  N{0} 

eona 

S.D. 
of  MC 

E  N{0} 

eona 

S.D. 
of  MC 

12.62 

25.24 

37.86 

126.21 

* 

13.01 

.03 

25.57 

.05 

38.20 

.08 

126.44 

.27 

15.91 

31.82 

47.73 

159.09 

* 

16.51 

.07 

32.32 

.14 

48.58 

.22 

159.61 

.68 

17.85 

35.69 

53.54 

178.45 

* 

18.44 

.09 

36.23 

.21 

54.53 

.30 

178.25 

.95 

20.00 

40.00 

60.00 

* 

200.01 

* 

20.98 

.13 

40.59 

.27 

60.56 

.40 

200.71 

.40 

25.05 

50.09 

75.14 

* 

250.47 

* 

26.62 

.20 

52.65 

.42 

76.00 

.60 

248.27 

1.91 

31.21 

62.42 

93.62 

* 

312.08 

j 

* 

34.54 

.32 

65.52 

.58 

93.93 

.84  ! 

315.47 

2.74 

40.72 

81.44 

122.16 

407.20 

* 

48.27 

.46 

89.65 

.87 

128.39 

1.22 

406.78 

3.91  | 

52.52 

105.04 

157.56 

525.21 

* 

72.75 

.71 

127.08 

1.25 

180.15 

1.80 

] 

533.15 

5.23 

83.93 

167.87 

251.80 

839.35 

189.58 

1.86 

315.25 

3.10 

428,10 

4.30 

1099.02 

10.96 

(In  Table  1,  TH  represents  the  theoretical  value  one  would  expect 
0 

for  E.N°  using  Theorem  1(11);  MC  represents  the  estimated  based  on  the 
U  A 

Monte  Carlo  trials.  The  (Monte  Carlo)  standard  deviation  of  this  esti¬ 
mate  is  given  under  the  heading  of  "S.D.  of  MC."  The  starred  cells 
in  Table  1  are  those  where  TH  -  MC  did  not  exceed  2  (Monte  Carlo) 
standard  deviations  of  TH.) 

{q} 

TABLE  2:  Ratios  of  asymptotic  theory  predictions  of  to  Monte 

Carlo  estimates  (TH/MC) 


X 

10 

20 

30 

100 

.4 

.97 

.99 

.99 

1.00 

.8 

.96 

.98 

.98 

1.00 

1.0 

.97 

.99 

.98 

1.00 

1.2 

.95 

.99 

.98 

1.00 

1.6 

.94 

.94 

.99 

1.01 

2.0 

.90 

.95 

1.00 

.99 

2.5 

.84 

.91 

.95 

1.00 

3.0 

.72 

.83 

.87 

.99 

4.0 

.44 

.53 

.59 

.76 
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The  results  show  a  surprisingly  good  fit,  even  for  low  values 


of  A  (as  long  as  6  Is  not  too  large) .  (Table  2  presents  the  ratio 
between  the  theoretical  value  of  TH  and  the  Monte  Carlo  estimate  MC.) 
It  seems  clear  that  for  most  practical  purposes  the  asymptotic  formula 
could  be  safely  applied.  (Shewhart  control  charts  using  "3a  limits" 

-  often  used  in  practice  -  have  a  Pg-expected  stopping  time  of  741.) 

For  an  indication  of  how  well  one  may  expect  the  formula  of 
Theorem  4  to  fit,  see  Poliak  and  Siegmund  (1975).  One  would  expect 
the  formula  presented  there  to  hold  as  well  as  the  formulae  presented 

here,  provided  that  yNA  is  large  enough  for  the  distribution  of 

k  {0} 

log [ 1  +  i  exp{-£i=1  ))  to  have  approximately  reached  its  limit¬ 

ing  distribution. 
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I 


VI.  REMARKS 


1.  In  Theorems  1,  3,  4  if  1(0)  =  it  is  possible  to  show 

{ft}  F 

that  Ertlr  /A  -*•  00  as  A  -*■  00  and  E,  N./log  A  +  0  as  A  +  *. 

U  A  1  A 

2.  Using  the  method  involved  in  showing  the  validity  of 
Remark  1,  one  can  show  that  Theorem  2  remains  valid  with 
F({y|Ky)  <  °°})  >  0. 

3.  It  seems  reasonable  to  conjecture  that  Theorem  2  remains 

(y) 

valid  if  the  -distribution  of  is  just  assumed  to  be  non-lattice. 
The  proof  given  above  for  Theorem  2  breaks  down  because  the  uniformity 
of  a  renewal-theoretic  convergence  used  in  the  proof  of  Lemma  1  need 
not  exist  if  the  strongly  non-lattice  assumption  is  dropped. 

4.  In  the  lattice  case,  even  a  version  of  Theorem  1  seems  to 


be  difficult  to  formulate.  Despite  X^'s  being  lattice,  R^  is  not,  and 

{0} 

the  proof  presented  here  -  which  conditions  on  N^yc  -  does  not  yield 
an  expression  for  the  non-lattice  part  of  the  asymptotic  P^-distribution 


*  i  Je) 

of  log  R^{e}- 
A 


log  A. 
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Suppose  one  is  able  to  observe  sequentially  a  series  of 


observations  X^i^i  **  *»  sucb  that  are 

l.i.d.  with  known  density  fg  and  .  are  i*i«d.  with  den¬ 

sity  fg  where  'J  is  unknown.  Define 

»<e>  .  ?  J  W 
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It  is  known  that  rules  which  call  for  stopping  and  raising  an  alarm 

{0} 

the  first  time  n  that  or  a  mixture  thereof  exceeds  a  prespecl- 

n 

fled  level  A  are  optimal  methods  of  detecting  that  the  density  of  the 
observations  is  not  f Q  any  more. 

Practical  applications  of  such  stopping  rules  require  know¬ 
ledge  of  their  operating  characteristics,  whose  exact  evaluation  is 
difficult.  Here  are  presented  asymptotic  (A  ”)  expressions  for  the 
expected  stopping  times  of  such  stopping  rules  (a)  when  V  -  “  and  (b) 
when  V  -  1.  He  assume  that  the  densities  fg  form  an  exponential 
family  and  that  the  distribution  of  log(fg(Xi>/f Q(Xt))  is  (strongly) 
non-lattice. 

Monte  Carlo  studies  indicate  that  the  asymptotic  expressions 
are  very  good  approximations  even  when  the  expected  sample  sizes  are 
small. 
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